Skip to content

Conversation

@Pouyanpi
Copy link
Collaborator

@Pouyanpi Pouyanpi commented Oct 1, 2025

Stack Info

This PR is part of a stack:

  1. refactor!: drop reasoning trace extraction logic #1427
  2. feat: add reasoning trace extraction from llm calls #1431
  3. feat: emit BotThinking events with reasoning traces #1432 (this PR)
  4. feat(logging): improve event logging and add bot thinking display #1434

⚠️ Depends on:

#1427#1431

Description

Extracts reasoning traces from LLM responses and emits BotThinking
events before BotMessage events. Adds bot_thinking_content to context
for use in output rails.

@Pouyanpi Pouyanpi changed the title Feat/bot thinking events feat: emit BotThinking events with reasoning traces Oct 1, 2025
@Pouyanpi Pouyanpi added this to the v0.18.0 milestone Oct 2, 2025
@tgasser-nv tgasser-nv self-requested a review October 6, 2025 14:58
@Pouyanpi Pouyanpi force-pushed the feat/reasoning-trace-extraction-langchain branch 2 times, most recently from 80e3466 to be7223e Compare October 13, 2025 07:59
Base automatically changed from feat/reasoning-trace-extraction-langchain to develop October 13, 2025 08:09
Extracts reasoning traces from LLM responses and emits BotThinking
events before BotMessage events. Adds bot_thinking_content to context
for use in output rails.
@Pouyanpi Pouyanpi force-pushed the feat/bot-thinking-events branch from 8ae50ef to 98c8911 Compare October 13, 2025 08:13
@NVIDIA-NeMo NVIDIA-NeMo deleted a comment from codecov-commenter Oct 13, 2025
@codecov-commenter
Copy link

Codecov Report

❌ Patch coverage is 74.19355% with 8 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
nemoguardrails/actions/llm/generation.py 73.33% 8 Missing ⚠️

📢 Thoughts on this report? Let us know!

@Pouyanpi Pouyanpi merged commit 59d1a6d into develop Oct 13, 2025
8 checks passed
@Pouyanpi Pouyanpi deleted the feat/bot-thinking-events branch October 13, 2025 09:04
tgasser-nv pushed a commit that referenced this pull request Oct 14, 2025
)

Extracts reasoning traces from LLM responses and emits BotThinking
events before BotMessage events. Adds bot_thinking_content to context
for use in output rails.

* feat: bot thinking in self check output rails

* docs(example): add self check output rails for bot thinking
tgasser-nv pushed a commit that referenced this pull request Oct 14, 2025
)

Extracts reasoning traces from LLM responses and emits BotThinking
events before BotMessage events. Adds bot_thinking_content to context
for use in output rails.

* feat: bot thinking in self check output rails

* docs(example): add self check output rails for bot thinking
tgasser-nv pushed a commit that referenced this pull request Oct 14, 2025
)

Extracts reasoning traces from LLM responses and emits BotThinking
events before BotMessage events. Adds bot_thinking_content to context
for use in output rails.

* feat: bot thinking in self check output rails

* docs(example): add self check output rails for bot thinking
Pouyanpi added a commit that referenced this pull request Oct 23, 2025
Refactor reasoning trace handling to expose reasoning content as a
separate
field in GenerationResponse instead of prepending it to response
content.

- Add reasoning_content field to GenerationResponse for structured
access
- Populate reasoning_content in generate_async when using
GenerationOptions
- Wrap reasoning traces in <thinking> tags for dict/string responses
- Add comprehensive tests for new field and behavior changes

This improves API usability by separating reasoning content from the
actual
response, allowing clients to handle thinking traces independently.

Resolves the TODO from PR #1432 about not prepending reasoning traces to
final generation content.
Pouyanpi added a commit that referenced this pull request Oct 23, 2025
BREAKING CHANGE: Reasoning traces are no longer prepended directly to
response content. When using GenerationOptions, reasoning is now
available
in the separate reasoning_content field. Without GenerationOptions,
reasoning is wrapped in <thinking> tags.

Refactor reasoning trace handling to expose reasoning content as a
separate
field in GenerationResponse instead of prepending it to response
content.

- Add reasoning_content field to GenerationResponse for structured
access
- Populate reasoning_content in generate_async when using
GenerationOptions
- Wrap reasoning traces in <thinking> tags for dict/string responses
- Update test to reflect new behavior (no longer prepending reasoning)
- Add comprehensive tests for new field and behavior changes

This improves API usability by separating reasoning content from the
actual
response, allowing clients to handle thinking traces independently.

Resolves the TODO from PR #1432 about not prepending reasoning traces to
final generation content.
Pouyanpi added a commit that referenced this pull request Oct 24, 2025
BREAKING CHANGE: Reasoning traces are no longer prepended directly to
response content. When using GenerationOptions, reasoning is now
available
in the separate reasoning_content field. Without GenerationOptions,
reasoning is wrapped in <thinking> tags.

Refactor reasoning trace handling to expose reasoning content as a
separate
field in GenerationResponse instead of prepending it to response
content.

- Add reasoning_content field to GenerationResponse for structured
access
- Populate reasoning_content in generate_async when using
GenerationOptions
- Wrap reasoning traces in <thinking> tags for dict/string responses
- Update test to reflect new behavior (no longer prepending reasoning)
- Add comprehensive tests for new field and behavior changes

This improves API usability by separating reasoning content from the
actual
response, allowing clients to handle thinking traces independently.

Resolves the TODO from PR #1432 about not prepending reasoning traces to
final generation content.
Pouyanpi added a commit that referenced this pull request Oct 24, 2025
…ending (#1468)

* feat(llm)!: add reasoning_content field to GenerationResponse

BREAKING CHANGE: Reasoning traces are no longer prepended directly to
response content. When using GenerationOptions, reasoning is now
available
in the separate reasoning_content field. Without GenerationOptions,
reasoning is wrapped in <thinking> tags.

Refactor reasoning trace handling to expose reasoning content as a
separate
field in GenerationResponse instead of prepending it to response
content.

- Add reasoning_content field to GenerationResponse for structured
access
- Populate reasoning_content in generate_async when using
GenerationOptions
- Wrap reasoning traces in <thinking> tags for dict/string responses
- Update test to reflect new behavior (no longer prepending reasoning)
- Add comprehensive tests for new field and behavior changes

This improves API usability by separating reasoning content from the
actual
response, allowing clients to handle thinking traces independently.

Resolves the TODO from PR #1432 about not prepending reasoning traces to
final generation content.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants